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ABSTRACT 


\ 

■~^Two  problems  with  the  multispectral  smoothing  technique 
of  Pietikainen  and  Rosenfeld  are  that  random  sampling  fluctua¬ 
tions  in  the  scatter  plot  may  produce  spurious  pixel  classes, 
and  that  noise  pixels  sufficiently  distinct  from  their  neigh¬ 
bors  will  not  be  smoothed  at  all.  These  problems  can  be  over¬ 
come  by  the  use  of  variable-neighborhood  smoothing  in  the 
scatterplot,  and  by  a  judicious  use  of  median  filtering  in 
conjunction  with  multispectral  smoothing. 
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1.  Introduction 


A  new  smoothing  technique,  which  uses  the  global  histogram 
of  an  image  to  guide  a  local  averaging  process,  was  proposed 
by  Narayanan  and  Rosenfeld  [3).  Recently,  Pietikainen  and 
Rosenfeld  [5]  have  generalized  this  technique  to  handle  multi- 
spectral  images.  The  substance  of  these  techniques  is  that  a 
pixel  is  averaged  with  only  a  subset  of  its  neighbors,  speci¬ 
fically  those  that  satisfy  both  the  following  conditions: 

Firstly,  the  gray  level  of  the  neighbor  should  occur  more  fre¬ 
quently  than  that  of  the  given  pixel.  Secondly,  there  should 
be  in  the  histogram  of  the  image  no  significant  concavity  between 
the  gray  level  of  the  neighbor  and  that  of  the  given  pixel.  (For 
a  multispectral  image,  the  scatterplot  of  one  band  against  ano¬ 
ther  is  used  instead  of  the  histogram.)  The  effect  of  this  is 
that  a  pixel  is  averaged  only  with  more  typical  (more  common) 
neighbors  of  the  same  population. 

In  practice,  the  histogram  (or  scatterplot)  of  an  image 
will  have  many  fluctuations  that  are  merely  artefacts  of  limited 
sampling.  These  fluctuations  produce  spurious  peaks  which  mis¬ 
lead  the  smoothing  process.  The  authors  mentioned  above  coped 
with  this  problem  by  uniformly  smoothing  the  histogram  (or 
scatterplot)  itself.  However,  it  has  been  remarked  [4,6]  that  in 
a  histogram  the  more  common  values  have  their  frequencies  estimated 
fairly  reliably,  while  it  is  the  rarer  values  that  are  more 


subject  to  sampling  errors.  This  has  lead  us  to  investigate 
the  effects  of  nonuniform  histogram  smoothing. 

Another  difficulty  with  the  techniques  mentioned  aoove 
is  that  a  noise  pixel,  if  sufficiently  distinct,  may  well  be 
judged  to  belong  to  a  separate  population  from  its  neighbors 
and  not  be  smoothed  at  all.  So  we  have  also  considered  some 
methods  for  removing  such  isolated  noise  spots. 
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2 .  Algorithm  and  experiments 

For  these  experiments,  we  used  the  same  smoothing  technique 
as  Pietikainen  and  Rosen f eld,  except  for  the  way  the  scatterplot 
is  smoothed.  They  averaged  the  frequency  of  a  point  in  the 
scatterplot  with  the  frequencies  in  a  3-by-3  neighborhood  in  the 
scatterplot.  We  instead  construct  progressively  larger  neigh¬ 
borhoods  around  the  point  until  the  total  frequency  within  one 
of  the  neighborhoods  reaches  or  exceeds  a  threshold  value  S. 

Thus  little  smoothing  is  done  in  those  regions  of  the  scatter¬ 
plot  where  frequencies  are  high,  and  more  smoothing  is  done 
where  they  are  low.  We  start  with  a  trivial  1  by  1  neighborhood 
(the  point  alone) ,  and  increase  the  side  of  the  neighborhood 
in  steps  of  two:  3  by  3,  5  by  5,  and  so  on.  We  call  this 
"optional"  scatterplot  smoothing  since  a  point  in  the  scatter¬ 
plot  with  sufficient  frequency  will  not  be  smoothed  at  all. 

We  also  used  a  variant  called  "obligatory"  scatterplot  smooth¬ 
ing,  in  which  we  start  the  series  of  neighborhoods  with  the 
3-by-3  size,  so  that  some  smoothing  is  always  performed  at 
every  point  in  the  scatterplot.  (Notice  that  the  neighborhood 
size  used  for  smoothing  the  histogram  is  quite  unrelated  to 
the  neighborhood  size  used  for  smoothing  in  the  image,  which 
in  this  report  is  always  3  by  3.) 


Figure  1  shows  the  red  and  green  bands  of  a  color  picture 
of  a  house  (gray  levels  0  to  63  in  each  band) ,  together  with 
the  64  by  64  scatterplot  of  red  against  green,  in  which  the 


frequencies  are  displayed  as  brightnesses,  after  both  linear 
and  logarithmic  scaling.  Figures  2  and  3  show  the  results  of 
5  and  10  iterations  of  PietikMinen ' s  method  without  any  smooth¬ 
ing  of  the  scatterplot.  Figures  4  and  5  show  the  results  of 
5  and  10  iterations  using  optional  scatterplot  smoothing  with 
threshold  value  S=300.  Notice  that  the  images  are  more  smoothed 
than  those  in  Figures  2  and  3,  and  that  the  scatterplots  are 
more  cleanly  separated  into  fewer  peaks.  For  comparison, 

Figur^  6  shows  the  result  of  10  iterations  with  a  different 
threshold  value,  S=500.  The  difference  between  Figures  5  and  6 
seems  minor,  although  the  higher  value  of  S  causes  more  smoothing 
of  the  scatterplot  and  image.  Figures  7  and  8  are  the 

same  as  4  and  5,  respectively,  except  that  obligatory  scatter¬ 
plot  smoothing  is  used.  The  difference  in  the  results  is  very 
slight.  Obligatory  smoothing  leaves  the  scatterplot  more  dif¬ 
fuse.  The  images  themselves  are  a  little  smoother,  although 
this  is  not  apparent  in  the  figures  here,  because  of  the  limited 
gray-level  resolution. 

Pietikainen' s  method  has  a  parameter  X,  which  is  a  threshold 
on  the  significance  of  concavities  in  the  scatterplot.  The  low¬ 
er  the  value  of  X,  the  deeper  a  concavity  needs  to  be  before  it  is 
considered  significant.  Figures  7  and  8  were  created  with 
X=0.1.  Figures  9  and  10  are  analogous,  wi  .h  X=1.0.  Figures  11 
and  12  are  the  same,  except  that  X=0.01.  It  can  be  seen  that 
the  high  value  of  X  reduces  the  smoothing,  because  the  scatterplot 


is  divided  into  too  many  peaks.  With  the  low  values,  only 
the  significant  peaks  are  respected.  So  the  smoothing  effect 
is  greater,  and  does  not  depend  so  much  on  the  precise  value 
of  X . 

Thus  we  can  exert  some  control  over  the  degree  of  smoothing 
by  adjusting  the  parameters  S  and  X,  and  by  the  selection  of 
optional  or  obligatory  smoothing.  However,  the  results  are  not 
unduly  sensitive  to  these  choices,  so  their  exact  settings 
are  not  critical.  We  have  chosen  to  use  X=0.1,  S=300.  and  obli¬ 
gatory  smoothing  for  all  the  images  in  this  report,  unless 
otherwise  stated. 

In  order  to  study  the  effectiveness  of  these  smoothing  tech¬ 
niques  on  very  noisy  images,  we  added  noise  to  our  house  image. 
Figure  13  shows  the  house  image  with  independent,  zero-mean 
Gaussian  noise  of  standard  deviation  o= 5  added  to  both  bands. 
Figures  14  and  15  show  the  results  of  5  and  10  iterations  of  our 
smoothing  technique.  The  noise  has  mostly  been  removed,  ex¬ 
cept  for  some  remaining  specks,  and  the  scatterplot  has  almost 
condensed  into  three  main  peaks.  We  applied  3  by  3  median  fil¬ 
tering  to  each  band  of  Figures  14  and  15;  the  results  appear 
in  Figures  16  and  17,  respectively.  The  remaining  noise  spots 
nave  been  almost  entirely  removed,  and  the  scatterplot  is  fur¬ 
ther  condensed,  although  some  of  the  fine  details  of  the  original 
image  have  been  obliterated.  We  also  tried  median  filtering 
before  smoothing.  Figure  18  shows  the  results  of  3  by  3  median 


filtering  on  both  bands  of  the  noisy  image  (Figure  13) . 

Figures  19  and  20  show  the  results  of  5  and  10  iterations  of 
smoothing  on  this  median-filtered  image.  The  results  are  similar 
to  those  obtained  by  applying  median  filtering  after  smoothing, 
but  not  quite  so  good.  Another  standard  technique  for  removing 
noise  is  mean  filtering.  We  therefore  tried  5  by  5  mean  fil¬ 
tering  on  our  noisy  picture  (Figure  21) ,  and  then  applied  10 
iterations  of  smoothing  (Figure  22) .  It  is  interesting  that 
the  smoothing  process  is  able  to  reconstruct  sharp  edges  — 
pixels  on  a  blurred  edge  are  forced  to  move  into  one  region  popu¬ 
lation  or  the  other.  However,  many  noise  spots  still  remain, 
since  mean  filtering  cannot  remove  all  traces  of  noise  spots. 

In  Figures  23  and  24  we  see  the  effects  of  5  and  10  iterations 
respectively  of  a  combined  process  of  smoothing  followed  by  3-by-3 
median  filtering  on  every  iteration.  A  lot  of  detail  has  been 
lost,  and  the  result  is  a  simplified,  cartoon-like  version  of 
the  original. 

As  suggested  by  Pietikainen  and  Rosenfeld  [5] ,  we  applied 
this  smoothing  technique  to  LANDSAT  images,  to  see  whether 
smoothing  would  result  in  any  improvement  in  classification,  as 
measured  by  comparison  with  the  ground  truth  [2] .  Unfortunately, 
no  consistent  improvement  was  found.  It  seems  that  in  LANDSAT 
images  at  least,  the  large  number  of  classes  present  in  each 
image  causes  confusion  of  populations  in  the  scatterplot,  leading 
to  misclassif ication. 


3.  Concluding  remarks 


The  use  of  the  global  distribution  of  pixel  values  in  an 
image  in  order  to  guide  a  smoothing  process  is  a  powerful 
and  effective  technique  for  smoothing  images  witnout  blurring 
edges.  In  this  report  we  have  investigated  ways  of  overcom¬ 
ing  two  problems  with  this  technique:  spurious  peaks  in  the 
image's  histogram  (or  scatterplot)  caused  by  random  sampling 
variation,  and  isolated  noise  spots  which  appear  to  belong  to 
a  different  population  from  their  neighbors  and  are  hence  not 
smoothed.  The  first  problem  can  be  overcome  by  non-uniform 
smoothing  of  the  scatterplot  (or  histogram) ,  and  the  second 
by  a  judicious  use  of  median  filtering.  Both  of  these  processes 
are  important  adjuncts  to  the  original  smoothing  technique,  espe¬ 
cially  for  noisy  images. 


References 


1.  L.  S.  Davis  and  A.  Rosenfeld,  "Noise  cleaning  by  iterated 
local  averaging,"  IEEE  Trans.  Syst.  Man  Cybern.,  vol.  8, 

1978,  pp.  705-710. 

2.  L.  S.  Davis,  C.  Y.  Wang,  and  H.  C.  Xie,  "Some  experiments 
in  multi-spectral,  multi-temporal  crop  classification 
using  relaxation  techniques,"  Computer  Science  Tech.  Report 
1131,  University  of  Maryland,  December  1981. 

3.  K.  A.  Narayanan  and  A.  Rosenfeld,  "Image  smoothing  by 
local  use  of  global  information,"  IEEE  Trans.  Syst.  Man 
Cybern. ,  vol.  11,  1981,  pp.  826-831. 

4.  P.  M.  Narendra  and  M.  Goldberg,  "A  non-parametric  cluster¬ 
ing  scheme  for  LANDSAT,"  Patt.  Recog.,  vol.  9,  1977, 

pp.  207-215.  '  " 

5.  M.  Pietikainen  and  A.  Rosenfeld,  "Multispectral  image 
smoothing  by  local  use  of  global  information,"  Computer 
Science  Tech.  Report  1113,  University  of  Maryland,  October 
1981. 

6.  S.  M.  Pizer  and  H.  G.  Vetter,  "Perception  and  processing 
of  medical  radioisotope  scans,"  pp.  147-156  in  Pictorial 
Pattern  Recognition,  G.  C.  Cheng,  R.  S.  Ledley,  D.  K.  Pollock 
and  A.  Rosenfeld,  eds.,  Thompson  Book  Company,  Washington,  DC 
1968. 

7.  A.  Rosenfeld  and  A.  C.  Kak,  Digital  Picture  Processing,  Aca¬ 
demic  Press,  New  York,  1976. 

8.  A.  Scher,  F.  R.  D.  Velasco  and  A.  Rosenfeld,  "Some  new 
image  smoothing  techniques,"  IEEE  Trans.  Syst.  Man  Cybern., 
vol.  10,  1980,  pp.  153-158. 


Figure  1.  Original  red 
(left)  and  green  (right) 
bands  of  house  picture, 
with  plain  (left)  and  loga¬ 
rithmic  (right)  scatterplot 
Following  figures  show  only 
logarithmic  scatterplots. 


Figure  2 .  A. 
of  Pietikain* 
(without  any 
smoothing,  X 


t  5  iteratioi 
s  smoothing 
.^tterplot 


Figure  3.  Same  as  Figure  21 
but  after  10  iterations. 


Figure  4 .  After  5  iterations  i 
with  optional  scatterplot  j 
smoothing,  S»300,  A=0.1.  j 


Figure  5.  Same  as  Figure  4, 
but  after  10  iterations. 


Figure  6.  After  10  iterations 
with  optional  scatterplot 
smoothing,  S=500,  A=0.1. 


Figure  7.  After  5  itera¬ 
tions  with  obligatory  scat 
terplot  smoothing,  S=300, 
X=0.1. 


Figure  8.  Same  as  Figure  7 
but  after  10  iterations. 


! 


i  Figure  11.  After  5 

iterations  with  obli¬ 
gatory  scatterplot 
smoothing,  S=300, 
A=0.01. 


Figure  12.  Same  as 
Figure  11,  but  after 
10  iterations. 


Figure  13.  Noisy  house 
picture,  a=5. 


Figure  14 .  After  5  itera¬ 
tions  smoothing  on  Figure 
13. 


■Figure  15.  After  10  itera 

tions  of  smoothing  on 
Figure  13. 


Figure  16.  After  3 
by  3  median  filtering 
on  Figure  14. 


Figure  17.  After  3 
by  3  median  filtering 
on  Figure  15. 


Figure  18.  After  3  by 
3  median  filtering  on 
Figure  13. 


Figure  19.  After  5  itera¬ 
tions  of  smoothong  on 
Figure  18. 


Figure  20.  After  10 
iterations  of  smoothing 
on  Figure  18. 


Figure  21.  After  5 
by  5  mean  filtering 
on  Figure  13. 


Figure  22.  After  10 
iterations  of  smooth¬ 
ing  on  Figure  21. 


Figure  23.  After  5 
iterations  of  combined 
smoothing  and  3  by  3 
median  filtering  on 
Figure  13. 


Figure  24.  Same  as 
Figure  23 r  but  after 
10  iterations. 
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